A. Field of the Invention
This invention relates to the field of data communications. Portions of the disclosure of this patent document contain material that is subject to copyright protection.
The copyright owner has no objection to the facsimile reproduction by anyone of the patent document or the patent disclosure as it appears in the Patent and Trademark Office file or records, but otherwise reserves all copyright rights whatsoever. Sun, Sun Microsystems, the Sun logo, Solaris, SPARC, xe2x80x9cWrite Once, Run Anywherexe2x80x9d, Java, JavaOS, JavaStation and all Java-based trademarks and logos are trademarks or registered trademarks of Sun Microsystems, Inc. in the United States and other countries.
B. Background Art
One use of the Internet is to deliver media information (such as a movie or sound file) to a user. A problem with current media delivery schemes is that it is difficult to deliver any other data while the media data is being sent. Sometimes it is desired to send media data and other data (like control data) at the same time, a process referred to as xe2x80x9cconcurrent deliveryxe2x80x9d. Right now there is no suitable scheme for concurrent delivery of media and control data. The problems associated with providing concurrent delivery of media data and control data can be understood by reviewing media delivery systems.
Media Delivery
In multimedia applications media data such as video or audio information is communicated from a server computer to a client computer (e.g. a user""s computer at home or office) via a xe2x80x9ctransport streamxe2x80x9d. A transport stream consists of a series of bursts or packets of media data. Before the media data is transmitted, it is first prepared by converting it into a suitable format for transmission. A number of schemes (e.g., Motion Picture Experts Group (MPEG) standards, such as MPEG-1, MPEG-2, MPEG-4, etc.) are available for preparing media data (e.g., audio, video, etc.) for transmission as a media transport stream.
Media data is sometimes referred to as xe2x80x9ctime awarexe2x80x9d or xe2x80x9ctime sensitivexe2x80x9d data. This is because it is important or required that data be available at a certain rate and/or at a certain time. For example, when viewing a video stream, it is important that new video frames be available as needed to prevent gaps or choppiness in displayed video data. If the data of a movie is being sent too slowly, for example, the displayed movie image may a freeze or go blank while the client computer waits for new movie data. Similarly, for audio data, new audio data must be provided continuously or there will be gaps during playback. These gaps may cause silence in the playback of the audio, causing the sound to be clipped or missing sounds and words. (Note that time aware data is not limited to media data. Time aware data is data that carries with it additional information representing timing relevant to the use of the data. For example, time aware data could include time stamps indicating deadlines by which the data should be processed).
In some circumstances, a control application is required, to help play back the media data or to control the playback of the media data. In some cases such a control application is an applet or an application (such as a Java(trademark) programming language applet or application). The applet or application is transmitted as bytecode from a server to a client and the client uses the applet or the application, for example, to control the playback of the media data. Bytecode or other serialized data or objects may need to be delivered to clients concurrently with the media information.
There is presently no effective mechanism available for delivery of bytecode instructions concurrently with media data. Typically, bytecode data is transported after all necessary media data is transmitted. That may be too late for some environments, where applets or control applications need to be processed as soon as they become available. One prior art solution is to send the control application first, prior to transmitting the media or time aware data. However, this leads to unwanted delays in playback of the media data. The user is expecting to see video data, but instead has to wait until the control data is received. This delay is undesirable.
Therefore, there is a need for transport of bytecode instruction concurrently with media data rather than prior to or subsequent to transport of the media information.
A method and apparatus for embedding of bytecode data in a transport stream is described. Embodiments of the invention serve to make Java(trademark) bytecode (in a class file) concurrently available to a user that is receiving media information through a transport stream. Bytecode information is embedded within spaces allocated in the transport stream, or in packets that carry the media data.
In one embodiment of the invention, media data is grouped in structured packets referred to as Packetized Elementary Stream (PES) packets. Bytecode data is embedded in predefined spaces in each PES packet. In one embodiment of the invention the pre-defined space is allocated within the header segment of a PES packet. In another embodiment bytecode instructions are embedded within a private stream segment of a PES packet.
In other embodiments, bytecode data is transmitted in the private stream segment and retransmission of bytecode data is accomplished in other regions of the PES packet. In another embodiment, bytecode data is transmitted in an adaptation field and retransmitted data is sent as PES header private data.
One transport stream used with the invention is an MPEG-2 transport stream that includes structured packets that transport the PES packets. These transport stream packets, in turn, have within them bytecode data embedded within spaces in the transport stream. In an embodiment of this invention bytecode can be embedded in a transport stream header or a transport stream payload segment.
In embodiments of the invention, the embedding of information under the methods of this invention is accomplished so that an MPEG-2 compliant decoder is able to use the stream irrespective of whether or not the decoder takes advantage of the embedded bytecode information. Further, the transport stream is transmitted with minimum changes in bit-rate in terms of both length and frequency and is within the allowed maximum bandwidth. Embodiments of this invention may also define tune-in points at which a decoder can start decoding and presenting bytecode information.